Repository navigation
fix(release): guard deferred fallback preparation and parallelize portfolio qualification - #9168
Conversation
…tfolio qualification Signed-off-by: LoopX Agent <[email protected]>
cc7808c to
1df7be9
Compare
loopx-agent
left a comment
There was a problem hiding this comment.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
Reviewer: model_agent · Gemini · Google (self_reported)
动机 / Motivation
受影响调用方与触发场景: 启用 hard_lease 或运行 v2.4.0 发布验收与真实模型行为评估的维护者和自动化调度流程。
改前与改后对比: 此前在作用域 user_gate 旁路选中就绪的延迟后继待办时,执行准备守卫误将其当成待执行任务送入租约投影,导致未开放状态被拒绝并清零运行资格;深路径下重规划写回拒收 JSON 超过 4096 字符而被截断;同时架构摘要清单、多 Goal 引导测试及独立测试导入与 main 脱节,实模组合只能串行执行。修复后,延迟后继重规划跳过租约准备并保留原有 open 回退的 hard_lease 校验,错误输出上限增至 8192 字符,测试与发布说明对齐,并新增默认关闭的 --max-workers 并发选项。
可观测收益: 在受保护的 SQLite 真实路径与单元测试中,deferred_successor_replan 保持 should_run=true 且不触发 selected_todo_not_open,open 作用域回退仍严格要求 claim_required;深路径 writeback rejection 恢复测试通过;run_actual_default_model_behavior_portfolio 在 max_workers=1 时保持顺序与回执完全一致,在 max_workers>1 时按目录顺序输出等价结果;v2.4.0.md 与 #9167/#9168 中英文发布说明同步。
本 PR 边界(Non-goals): 本 PR 不改动 execution_preparation.ts 的状态机规则,不改变默认 max_workers=1 顺序执行行为,也不在无凭证或隔离工作树中自动运行在线 Ark 接口。
若放任主干现状,v2.4.0 在启用 hard_lease 的真实施工环境中会把受 user_gate 旁路选出的就绪延迟后继误判阻断,深路径环境下的重规划拒收恢复链路会持续丢包截断,独立单文件测试收集与多 Goal 引导回归也会继续报错,同时 22 场景重复 2 次的真实模型组合评估耗时过长。此切片范围完整、可直接回归验证且低风险可逆。
改动思路 / Approach
本次改动遵循最小修复与现有职责归属原则(exact head 1df7be9328abcdc4ae4b0335b2dd718547a271a9):
- 延迟后继重规划与执行租约边界分离:在
apply_selected_todo_execution_preparation入口,当effective_action == "successor_replan_required"(复用AgentScopeFrontierAction.SUCCESSOR_REPLAN_REQUIRED)或scoped_user_gate_fallback.selected_executable.fallback_kind == "deferred_successor_replan"时,不再将其视为scoped_fallback_admitted,从而不向 TypeScriptprojectExecutionPreparation发起针对status="deferred"待办的租约查询。原有的open作用域回退仍照常进入hard_lease准备并强制claim_required。 - 宿主工具错误输出截断上限提升:将
replan_semantic_action_behavior.py中_HostToolError的输出截断常量由魔法数字4096抽取并提升为_MAX_HOST_TOOL_ERROR_OUTPUT_CHARS = 8192,保证较深临时目录路径下的replan_writeback_rejection_v0完整 JSON(含context_recovery.select_command)不被腰斩。 - Opt-in 并发组合执行:
run_actual_default_model_behavior_portfolio与scripts/qualify-doubao-model-behavior-live.py新增默认值为1的max_workers/--max-workers。max_workers == 1走原顺序分支;max_workers > 1使用有界ThreadPoolExecutor与threading.Lock保护aborted状态和全局计数集合,并借pool.map保留_SCENARIOS的确定性输出顺序。
四项运行时与测试改动各自停留在既有职责边界内:作用域延迟重规划过滤留在 Python 适配层 apply_selected_todo_execution_preparation,错误输出截断上限留在 replan_semantic_action_behavior 的宿主循环,并发执行留在真实模型评估组合及 live 脚本入口,不引入新的状态存储、协议字段或平行决策层。 当前 PR 仅修复已暴露的发布验收回归并增加 opt-in 并发参数,保留 TypeScript execution_preparation 对 open Todo 的既有语义、sequential 默认值(max_workers=1)与现有词汇表。
| 场景 / 路径 | 改前行为(origin/main) |
改后行为(1df7be9328ab) |
保护边界 |
|---|---|---|---|
scoped_user_gate_fallback + deferred_successor_replan |
误判 scoped_fallback_admitted=True,被 TS 拒为 selected_todo_not_open(should_run=False) |
排除租约准备,保持 should_run=True 与 successor_replan_required |
open 作用域回退仍走 hard_lease 准备 |
深路径 _HostToolError 返回 |
exc.output[:4096] 截断 replan_writeback_rejection_v0 JSON |
提升至 8192 字符(_MAX_HOST_TOOL_ERROR_OUTPUT_CHARS) |
非错误正常响应路径不变 |
| 真实模型组合评估执行 | 仅串行执行(22 * 2 次调用) |
默认 max_workers=1 保持全等;>1 有界线程池并发且保持顺序 |
禁止 bool 及 <1 参数 |
具体改动 / Changes
- 规范依据说明(
spec_basis):本 PR 针对发布前发现的回归缺陷与测试工具并发优化,无独立新增 RFC(no_spec);其中docs/product/releases/v2.4.0.md对照docs/product/release-readiness.md的双语发布说明契约校验通过,execution_preparation.py对照既有execution_preparation.ts与agent_scope.py状态契约。
关键代码讲解
-
apply_selected_todo_execution_preparation(loopx/control_plane/quota/execution_preparation.py:448-475)- 职责:在
should_run=True时为正常交付或开放态scoped_user_gate_fallback注入execution_preparation(hard_lease就绪/认领门禁)。 - 修复逻辑:读取
fallback = payload.get("scoped_user_gate_fallback")及其selected_executable,在scoped_fallback_admitted判定中新增两个收敛条件:payload.get("effective_action") != AgentScopeFrontierAction.SUCCESSOR_REPLAN_REQUIRED.value与fallback_selected.get("fallback_kind") != "deferred_successor_replan"。两者中任意一项成立即退出租约准备,避免把仅需生命周期重规划(reopen / supersede / no-follow-up)的deferred待办送入只接受open待办的 TypeScriptprojectExecutionPreparation。 - 保留边界:当选中项是普通
open作用域回退(fallback_kind非deferred_successor_replan且effective_action非successor_replan_required)时,依然触发project_selected_todo_execution_preparation,并在缺少活跃租约时返回claim_required。
- 职责:在
-
run_actual_default_model_behavior_portfolio(loopx/control_plane/testing/actual_default_model_behavior_portfolio.py:1393-1509)- 职责:编排 22 个真实模型行为场景(每场景 2 轮重复)及 6 项不变性/敏感性对比(contrasts)。
- 改动细节:先对
max_workers做严格类型校验(拒绝bool、非int与< 1);抽出_aborted_scenario_result与_run_one消除重复逻辑。max_workers == 1完全走原有串行逻辑;max_workers > 1时创建上限为min(max_workers, len(_SCENARIOS))的ThreadPoolExecutor,在_worker内通过threading.Lock读写aborted,利用pool.map(_worker, _SCENARIOS)保证返回的results与_SCENARIOS顺序一致,并在收集循环中仅对非"not_run"的场景写入observations和累加actor_call_count。
-
_run_qualification_loop(loopx/control_plane/testing/replan_semantic_action_behavior.py:57, 1760-1766)- 职责:驱动带真实 CLI / 沙箱 Shell 的模型行为验收循环。
- 改动细节:引入常量
_MAX_HOST_TOOL_ERROR_OUTPUT_CHARS = 8192,替换原来硬编码的4096,避免在较深tmp_path(如多级目录叠加 64 字符后缀)下refresh-state返回的replan_writeback_rejection_v0载荷在context_recovery.select_command字段前被截断。
-
main与counted_transport(scripts/qualify-doubao-model-behavior-live.py:60-173)- 职责:真实 Ark Doubao 模型行为评估的 CLI 入口与请求统计包装。
- 改动细节:新增
--max-workers(默认1,< 1时抛ValueError),并用transport_lock = threading.Lock()保护共享的provider_call_count与provider_models集合。各场景和重复轮次通过sha256(run_id)[:16]隔离临时夹具目录,避免并发写盘冲突。
其他对齐改动
- 摘要消费者清单与导入路径:
tests/architecture/test_content_digest_single_owner.py将新增使用BARE_SHA256_PATTERN的loopx.cli_commands.quota_capture和loopx.extensions.lark.private_delivery_discovery纳入CONSUMER_MODULES并移除已迁出的loopx.extensions.lark.private_conversations;tests/control_plane/test_effect_turn_live_quota_decision.py将canonical_authority_fixture改为完整包路径tests.control_plane.canonical_authority_fixture,恢复tests/test_host_launch_authority.py独立收集与执行。 - Smoke 与测试覆盖:
examples/bootstrap-command-pack-smoke.py对齐 PR #9152 在多 Goalgoal_selection_required分支中新增的"host_surface": None, "activation_method": None;test_quota_execution_preparation.py、test_actual_default_model_behavior_portfolio.py、test_replan_semantic_action_behavior.py均新增对应正反例与回归测试。 - 发布说明双语同步:
docs/product/releases/v2.4.0.md同步补充#9167(回环中继 401 恢复)与#9168(延迟后继排除租约准备、--max-workers、8 KiB 错误输出上限、摘要清单与引导 smoke/导入对齐)的英文与中文条目。
对主干的风险 / Risks
- Opt-in 并发隔离与反事实校验(
default_off_isolation&scope_coverage):max_workers=1(默认值)保持既有顺序执行流,不启用ThreadPoolExecutor,输出回执字节级等价;max_workers=4在测试与故障注入反事实(一个场景抛出RuntimeError)下均保持结果顺序确定性、not_run跳过统计正确且无锁竞争或崩溃。需注意:在max_workers > 1并发模式下若某一场景发生actor_error,已在线程池中开始执行的同批在途场景会跑完自身当前轮次,之后尚未启动的场景才会观测到aborted=True并标记为not_run(这是有界并发的合理语义,对串行模式无影响)。- 在真实 SQLite
hard_lease隔离夹具上执行了成对反事实校验:当作用域user_gate选中deferred_successor_replan时,quota should-run保持should_run=True、effective_action="successor_replan_required"且不生成execution_preparation;当同一 Todo 通过todo update --status open变为open回退时,仍准确触发execution_preparation.state == "claim_required"(task_lease_missing),未破坏开放任务的硬租约门禁。
- 线程安全边界检查:
- 核查了 7 个行为 actor(
DoubaoModelBehaviorActor、DoubaoOnboardingModelBehaviorActor、DoubaoSelectedTodoToolBehaviorActor、DoubaoReplanSemanticActionBehaviorActor、DoubaoScopedGateSuccessorToolBehaviorActor、DoubaoCapabilityMonitorRepairToolBehaviorActor、DoubaoTerminalSettlementToolBehaviorActor)、VisionShellHost、execute_loopx_cli与effect_runtime.py:actor 实例无可变共享状态,每个run_id使用独立临时目录、AF_UNIX套接字和子进程环境,TS Effect Runtime 通过指纹文件锁安全复用。
- 核查了 7 个行为 actor(
- 语义对齐与本地验证矩阵:
- 词汇表复用(
reuse_existing):未新增任何枚举、状态字面量或配置键,直接复用AgentScopeFrontierAction.SUCCESSOR_REPLAN_REQUIRED与既有的fallback_kind == "deferred_successor_replan"。 - 隔离工作树内
examples/semantic-vocabulary-drift-smoke.py因未安装node_modules(typescript开发依赖缺失)而退出失败(NPM_DEV_DEPENDENCIES_MISSING);鉴于本 PR 未改动任何.ts文件或语义词汇定义文件,该环境限制不阻塞评审结论。其余已执行验证全部 PASS(见下表):
- 词汇表复用(
| 验证项 | 状态 | 观察结果 |
|---|---|---|
10 个核心及关联 pytest 模块(pytest -q -n 8) |
PASS | 605 passed in 142.17s(含新增 3 个回归测试及 test_host_launch_authority.py 独立收集) |
3 项新增回归用例定向复跑(pytest -v -k ...) |
PASS | deferred_replan 排除、max_workers=4 等价性、深路径恢复均通过 |
TypeScript 单测(execution_preparation, interaction_contract, scoped_gate_fallback) |
PASS | 44/44 passed(node --experimental-strip-types --test) |
examples/bootstrap-command-pack-smoke.py(隔离 HOME) |
PASS | bootstrap command pack smoke passed |
examples/release/release-readiness-doc-smoke.py(12 个可选能力面) |
PASS | release-readiness-doc-smoke ok |
ruff check(全部变更 Python 文件)& mypy(配置严格基线 17 文件) |
PASS | 0 errors |
scripts/check_rfc_bilingual.py & loopx check --scan-path(11 文件) |
PASS | 双语 RFC 检查与公开边界扫描 0 errors |
loopx canary premerge --from-git-diff --git-diff-base origin/main |
FAIL(环境依赖) | 18/19 通过;仅 semantic-vocabulary-drift-smoke.py 因无 node_modules 报错 |
真实 SQLite hard_lease 正/反对比 & max_workers=4 异常中止反事实 |
PASS | deferred 跳过租约、open 仍需 claim_required、并发中止无异常 |
- 轻微非阻塞说明:
docs/product/releases/v2.4.0.md中将8192字符上限写为8 KiB(在纯 ASCII JSON 载荷下数值等价,对多字节非 ASCII 字符略偏口语化);_bound_semantic_writeback以外的terminal_settlement_tool_behavior.py仅对workspace_shell保留4096截断且不走refresh-stateJSON 输出,不影响恢复链路。
我的整体评价 / Verdict
- 长程与体验影响(
outcome_impact):long_horizon判定为preserved(修复了作用域网关旁路下延迟后继被错误阻断和深路径重规划恢复被截断的回归,同时保留开放任务的hard_lease防线);user_experience判定为improved(提供 opt-in--max-workers并发加速评估并修复独立测试收集及引导 smoke 漂移)。本 PR 仅涉及控制面 Python 逻辑、测试工具与发布文档,未修改任何渲染界面,因此visual_review判定为not_applicable(checked scope: full diff across 11 files,unverified_scope: none)。 - 代码体量、架构适配与语义结论:变更规模克制(+220/-42 across 11 files),无多余兼容分支或重复状态,架构归属评估(
architecture_assessment)为retain——四项运行时与测试改动各自停留在既有职责边界内:作用域延迟重规划过滤留在 Python 适配层 apply_selected_todo_execution_preparation,错误输出截断上限留在 replan_semantic_action_behavior 的宿主循环,并发执行留在真实模型评估组合及 live 脚本入口,不引入新的状态存储、协议字段或平行决策层。 当前 PR 仅修复已暴露的发布验收回归并增加 opt-in 并发参数,保留 TypeScript execution_preparation 对 open Todo 的既有语义、sequential 默认值(max_workers=1)与现有词汇表。 语义对齐上,PR 完全reuse_existing(复用既有AgentScopeFrontierAction.SUCCESSOR_REPLAN_REQUIRED与deferred_successor_replan,无新词汇引入)。 - 残留风险:实网 Ark API 在
--max-workers > 1下的限流表现未在本次本地无密钥隔离审查中实测;隔离工作树缺少node_modules导致semantic-vocabulary-drift-smoke.py的 TypeScript AST 扫描未在本地跑通(人工审查确认无.ts/词汇变更)。 - 精确 Head 确认:已于发布前核验精确提交
1df7be9328abcdc4ae4b0335b2dd718547a271a9,本地核心验证完备且未发现阻塞主干合并的缺陷。
English verdict: APPROVE at exact head 1df7be9. All 11 changed files cleanly fix the scoped deferred-replan execution-preparation regression, deep-path host-tool rejection truncation, digest manifest/test-import/smoke drift, and v2.4.0 release-note reconciliation while keeping --max-workers=1 default-off isolation and open-fallback hard-lease guarantees intact. Verified with 605 passed pytest cases, 44 passed TS tests, isolated bootstrap and 12-surface release-readiness smokes, ruff/mypy/public-boundary checks, and paired SQLite hard-lease counterfactuals (premerge canary semantic-vocabulary-drift-smoke.py failed only due to absent node_modules in the isolated worktree; no .ts or vocabulary changes in PR).
Summary
Fixes the remaining v2.4.0 exact-source qualification regressions surfaced on main and adds opt-in
--max-workersconcurrency to live Doubao portfolio qualification:loopx/control_plane/quota/execution_preparation.py): Excludeeffective_action == "successor_replan_required"andfallback_kind == "deferred_successor_replan"fromscoped_fallback_admittedinapply_selected_todo_execution_preparation, preventingexecution_preparation.tsfrom rejecting readydeferredlifecycle-replan fallbacks withselected_todo_not_openwhile preserving hard-lease preparation for openscoped_user_gate_fallbackdeliveries.loopx/control_plane/testing/replan_semantic_action_behavior.py): Raise_MAX_HOST_TOOL_ERROR_OUTPUT_CHARSfrom4096to8192soreplan_writeback_rejection_v0JSON payloads (exceeding 4 KB under deep pytesttmp_paths) are not truncated mid-JSON beforecontext_recovery.select_command.tests/architecture/test_content_digest_single_owner.py,examples/bootstrap-command-pack-smoke.py,tests/control_plane/test_effect_turn_live_quota_decision.py): Pinloopx.cli_commands.quota_captureandloopx.extensions.lark.private_delivery_discovery(removing retiredloopx.extensions.lark.private_conversations), align multi-goalhost_loop_activationsmoke expectation with PR fix(release): stabilize CLI help, architecture guards, onboarding gate, and public smokes #9152 (host_surface: null, activation_method: null), and package-qualifytests.control_plane.canonical_authority_fixturesotests/test_host_launch_authority.pycollects standalone.loopx/control_plane/testing/actual_default_model_behavior_portfolio.py,scripts/qualify-doubao-model-behavior-live.py): Addmax_workers: int = 1and--max-workers(default1preserves byte-identical sequential behavior;>1runs independent scenarios across a boundedThreadPoolExecutorwith lock-protected abort/counters while preserving_SCENARIOScatalog receipt order).docs/product/releases/v2.4.0.md): Reconcile EN/ZH notes with#9167and#9168.Verification (run in clean worktree)
pytest -q -n 8across all 9 affected/adjacent test modules (test_host_launch_authority.py,test_effect_turn_live_quota_decision.py,test_scoped_gate_successor_tool_behavior.py,test_actual_default_model_behavior_portfolio.py,test_quota_execution_preparation.py,test_replan_semantic_action_behavior.py,test_content_digest_single_owner.py,test_scoped_gate_fallback_cli.py,test_settled_monitor_user_gate.py): 591 passed, 0 failed.examples/bootstrap-command-pack-smoke.py: passed.examples/release/release-readiness-doc-smoke.py --release-notes docs/product/releases/v2.4.0.mdacross all 12 surfaces: passed.ruff check,mypy,scripts/check_rfc_bilingual.py --base origin/main --staged, andloopx checkacross all 11 changed files: all passed (0 errors, 0 warnings).